Signals and Meanings of Gaze in Animated Faces

The creation of conversational agents capable of expressive and communicative behaviors requires to define the relationship between the agent’s communicative intentions and how these intentions are expressed in a coordinated verbal and nonverbal message. Suppose an agent S (a Sender) has the goal of communicating something to an interlocutor A (Addressee) in a particular situation and context; s/he has to decide what to say, which words to employ, which intonation, gestures and facial expressions to display in the various phases of the message. We are currently working on the automatic generation of verbal and nonverbal messages in order to animate a 3D agent. Our plan is to define nonverbal communicative acts in the same type of structure as is used for verbal communicative acts. Several problems need to be solved: how natural language generation systems may be extended to include the generation of nonverbal communicative acts; how planning operators should be refined to include them; and finally how verbal and nonverbal communicative acts may be synchronized. In this work, we restrict ourselves to only one aspect of the visual display of communicative acts, leaving aside body posture, hand gestures and so on. We focus on gaze behavior and propose a meaning-to-face approach, aimed at simulating automatic generation of face expressions driven by semantic data. The dynamic aspects of the generation of meanings in the flowing of discourse which we do not deal in this paper, are addressed in other works [Poggi et al., 2000].